The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Hi @ydshieh - FYI this change appears to have broken some unit tests in DeepSpeed, specifically where we download a model. I'll look at ways to resolve this, I assume it is an issue in multiprocessing/threading.

@loadams Thank you for informing us. Hope there is a way that could work in both cases 🙏 Let me know if there is anything we can help here.

@ydshieh I'm also experiencing issues here for Sentence Transformers/Cross Encoder models, see UKPLab/sentence-transformers#3129

In short: multiprocessing.Process never works when not inside of __name__ == "__main__". I recognize that most programs should be using that line, but I'd rather not force it on my users.

If one of my users loads any model that only has a pytorch_model.bin, then it'll fail, e.g.:

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("embaas/sentence-transformers-gte-base")

or

from sentence_transformers import CrossEncoder

model = CrossEncoder("cross-encoder/ms-marco-MiniLM-L-6-v2")

which internally call

from transformers import AutoModel

model = AutoModel.from_pretrained("embaas/sentence-transformers-gte-base")

or

from transformers import AutoModelForSequenceClassification

model = AutoModelForSequenceClassification.from_pretrained("cross-encoder/ms-marco-MiniLM-L-6-v2")

All of these get:

RuntimeError:
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.

        To fix this issue, refer to the "Safe importing of main module"
        section in https://docs.python.org/3/library/multiprocessing.html

• Tom Aarsen

Hi @ydshieh - FYI this change appears to have broken some unit tests in DeepSpeed, specifically where we download a model. I'll look at ways to resolve this, I assume it is an issue in multiprocessing/threading.

Hi @loadams FYI: it's reverted back to use Thread in #35236 (as there are other issues reported)

should be using that line, but I'd rather not force it on m

@ydshieh - thanks for the update!